The present invention relates to calibration of a user-wearable multi-camera system. More specifically, this invention relates to calibration that includes a pose of the multi-camera system with respect to the user.
Augmented vision, also referred to as augmented reality or augmented reality vision, augments a user's view of the real world with superimposed computer generated graphical information. This information may be include, for example, a text label attached to some object in the scene, or a three-dimensional (3D) model of a patient's brain, derived from an MRI scan, and aligned to the real view of the person's head.
The user may observe the real world directly with his or her eyes, and the additional graphics information is blended in via a semi-transparent display located between the observer and the real scene. Such a display device can, for example, be an optical see-through head mounted display. The display can also be opaque, like a computer screen or a non-see-through head mounted display. It then presents to the user the complete augmented view, a combination of the real-world view and the graphics overlay. A video camera takes the place of the real-world observer to capture the real world-view. Two cameras may be implemented for stereo vision. A computer may be used to combine the live video with the graphics augmentation. A display device of this kind is, for example, a video-see-through head-mounted display.
The graphics are positioned, oriented, and scaled, or even rendered in a perspective fashion for correct alignment with the real-world view. To achieve precise alignment of the real and virtual view, the graphics may be anchored to a real-world object. For this knowledge of the position and orientation of the user's viewpoint is needed with respect to this object, and the orientation of the object. Thus, the relationship between two coordinate systems needs to be defined, one attached to the user's head, the other attached to the object.
Tracking denotes the process of keeping track of this relationship. Commercial tracking systems are available based on optical, magnetic, ultrasound, and mechanical means.
Calibration is needed to achieve correct alignment between virtual graphics objects and real objects in the scene. Calibrating a video-see-through head-mounted-display (HMD) can be done in an objective way, independent of a user, as real and virtual images are combined in the computer. In contrast, with an optical-see-through HMD the combination of the real and virtual images takes place finally in the user's eye, and the position of the user's eye behind the semi-transparent screen has critical influence on the alignment.
Different methods for calibrating an optical-see-through HMD are known as prior art. All known calibration methods require the user to align virtual structures with real reference structures. For example, in one method the user is shown a sequence of fixed graphical markers on the display and moves the head to bring them into alignment with a reference marker in the real scene.
In many applications one does not want a HMD in a wearable camera frame. One requirement is to accurately correlate a gaze from a wearer and a position of a camera worn by the wearer with a coordinate or a location in space. For vision applications needing both precise measurements and comfortable use, no known system currently exists that calibrates the pose of the head-worn multi-camera both to the external world and to the user, in particular to the eyeball position and orientation.
Registration and/or calibration systems and methods are disclosed in U.S. Pat. Nos. 7,639,101; 7,190,331 and 6,753,828. Each of these patents are hereby incorporated by reference.
Accordingly, a need exists for improved and novel systems and methods for calibration of a head-worn multi-camera both to the external world and to the user, in particular to the eyeball position.